Transition points in an image sequence

ABSTRACT

Techniques are proposed for embedding transition points in media content. A transition point system retrieves a time marker associated with a point of interest in the media content. The transition point system identifies a first position within the media content corresponding to the point of interest. The transition point system embeds data associated with the time marker into the media content at a second position that is no later in time than the first position. The transition point system causes a client media player to transition from a first image quality level to a second quality level based on the time marker.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of co-pending U.S. patent applicationSer. No. 13/958,268, filed Aug. 2, 2013, which is herein incorporated byreference in its entirety.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to the field of streaming media and, inparticular, to transition points in an image sequence.

Description of the Related Art

Streaming media often involves delivering some combination of audio,video, and text in a continuous manner from a content provider to aclient media player via a communications channel. Examples of streamingmedia include video, audio, closed caption text, and movies that includevideo, audio, and closed caption text in a single media stream. Theclient media player presents the streaming media to an end-user via asuitable playback mechanism. In general, streaming media, such as amovie, is viewed at a target frame rate that is more or less constant,where the target frame rate is the number of image frames presented perunit of time, such as 60 frames per second. The streaming media iscompressed and encoded so as to provide an acceptable image qualitylevel at the desired frame rate, within the confines of the availablebandwidth of the communications channel.

During playback of the streaming media, the bandwidth of thecommunications channel that is available for playback of the streamingmedia may vary dynamically. Such dynamic changes of available bandwidthmay result from a various causes, including increases or decreases inthe throughput of the communications channel, other demands on thecommunications channel, download requests or streaming media associatedwith other client media players, and the rendering demands andcapability of the client media player. Generally, delivering streamingmedia at a constant frame rate with a variable quality level isconsidered preferable to delivering streaming media at a variable framerate with a constant quality level. Accordingly, if the availablebandwidth decreases, then the content provider may deliver streamingmedia at a higher compression level, corresponding to lower bandwidthconsumption at a lower image quality level. If the available bandwidthincreases, then the content provider may deliver streaming media at alower compression level, corresponding to higher bandwidth consumptionat a higher image quality level. The image quality level of thestreaming media dynamically increases or decreases, responsive todynamic changes in the available bandwidth of the communicationschannel.

One drawback with this approach is that changes in image quality leveloccur based on available bandwidth of the communication channel withoutregard to the quality or continuity of the viewing experience over time.For example, a change in image quality level could occur at aparticularly noticeable point during a movie or during a particularlyimportant scene, resulting in a compromised viewing experience for theend-user.

SUMMARY OF THE INVENTION

One embodiment of the present invention includes a method for embeddingtransition points in media content. The method includes retrieving atime marker associated with a point of interest in the media content.The method further includes identifying a first position within themedia content corresponding to the point of interest. The method furtherincludes embedding data associated with the time marker into the mediacontent at a second position that is no later in time than the firstposition. The method further includes causing a client media player totransition from a first image quality level to a second quality levelbased on the time marker.

Other embodiments include, without limitation, a computer-readablestorage medium that includes instructions that enable a processing unitto implement one or more aspects of the disclosed methods as well as acomputing system configured to implement one or more aspects of thedisclosed methods.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the inventioncan be understood in detail, a more particular description of theinvention, briefly summarized above, may be had by reference toembodiments, some of which are illustrated in the appended drawings. Itis to be noted, however, that the appended drawings illustrate onlytypical embodiments of this invention and are therefore not to beconsidered limiting of its scope, for the invention may admit to otherequally effective embodiments.

FIG. 1 is a block diagram of a system configured to implement one ormore aspects of the invention;

FIG. 2 illustrates a layout of movie clips associated with transitionpoints, according to one embodiment of the present invention;

FIG. 3 illustrates a transition point system as may be implemented onthe computing system of FIG. 1, according to one embodiment of theinvention;

FIG. 4 sets forth a flow diagram of method steps for embeddingtransition points in media content with the system of FIG. 3, accordingto one embodiment of the invention;

FIG. 5 illustrates a content server configured to transmit a mediastream as may be implemented on the computing system of FIG. 1,according to one embodiment of the invention; and

FIGS. 6A-6B set forth a flow diagram of method steps for transmitting amedia stream that includes transition points with the system of FIG. 5,according to one embodiment of the invention.

DETAILED DESCRIPTION

In the following, reference is made to embodiments of the invention.However, it should be understood that the invention is not limited tospecific described embodiments. Instead, any combination of thefollowing features and elements, whether related to differentembodiments or not, is contemplated to implement and practice theinvention. Furthermore, although embodiments of the invention mayachieve advantages over other possible solutions and/or over the priorart, whether or not a particular advantage is achieved by a givenembodiment is not limiting of the invention. Thus, the followingaspects, features, embodiments and advantages are merely illustrativeand are not considered elements or limitations of the appended claimsexcept where explicitly recited in a claim(s). Likewise, reference to“the invention” shall not be construed as a generalization of anyinventive subject matter disclosed herein and shall not be considered tobe an element or limitation of the appended claims except whereexplicitly recited in a claim(s).

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

Hardware Overview

FIG. 1 depicts one architecture of a system 100 within which embodimentsof the present invention may be implemented. This figure in no waylimits or is intended to limit the scope of the present invention.

System 100 may be a personal computer, video game console, personaldigital assistant, rendering engine, or any other device suitable forpracticing one or more embodiments of the present invention.

As shown, system 100 includes a central processing unit (CPU) 102 and asystem memory 104 communicating via a bus path that may include a memorybridge 105. CPU 102 includes one or more processing cores, and, inoperation, CPU 102 is the master processor of system 100, controllingand coordinating operations of other system components. System memory104 stores software applications and data for use by CPU 102. CPU 102runs software applications and optionally an operating system. Memorybridge 105, which may be, e.g., a Northbridge chip, is connected via abus or other communication path (e.g., a HyperTransport link) to an I/O(input/output) bridge 107. I/O bridge 107, which may be, e.g., aSouthbridge chip, receives user input from one or more user inputdevices 108 (e.g., keyboard, mouse, joystick, digitizer tablets, touchpads, touch screens, still or video cameras, motion sensors, and/ormicrophones) and forwards the input to CPU 102 via memory bridge 105.

A display processor 112 is coupled to memory bridge 105 via a bus orother communication path (e.g., a PCI Express, Accelerated GraphicsPort, or HyperTransport link); in one embodiment display processor 112is a graphics subsystem that includes at least one graphics processingunit (GPU) and graphics memory. Graphics memory includes a displaymemory (e.g., a frame buffer) used for storing pixel data for each pixelof an output image. Graphics memory can be integrated in the same deviceas the GPU, connected as a separate device with the GPU, and/orimplemented within system memory 104.

Display processor 112 periodically delivers pixels to a display device110 (e.g., a screen or conventional CRT, plasma, OLED, SED or LCD basedmonitor or television). Additionally, display processor 112 may outputpixels to film recorders adapted to reproduce computer generated imageson photographic film. Display processor 112 can provide display device110 with an analog or digital signal.

A system disk 114 is also connected to I/O bridge 107 and may beconfigured to store content and applications and data for use by CPU 102and display processor 112. System disk 114 provides non-volatile storagefor applications and data and may include fixed or removable hard diskdrives, flash memory devices, and CD-ROM, DVD-ROM, Blu-ray, HD-DVD, orother magnetic, optical, or solid state storage devices.

A switch 116 provides connections between I/O bridge 107 and othercomponents such as a network adapter 118 and various add-in cards 120and 121. Network adapter 118 allows system 100 to communicate with othersystems via an electronic communications network, and may include wiredor wireless communication over local area networks and wide areanetworks such as the Internet.

Other components (not shown), including USB or other port connections,film recording devices, and the like, may also be connected to I/Obridge 107. For example, an audio processor may be used to generateanalog or digital audio output from instructions and/or data provided byCPU 102, system memory 104, or system disk 114. Communication pathsinterconnecting the various components in FIG. 1 may be implementedusing any suitable protocols, such as PCI (Peripheral ComponentInterconnect), PCI Express (PCI-E), AGP (Accelerated Graphics Port),HyperTransport, or any other bus or point-to-point communicationprotocol(s), and connections between different devices may use differentprotocols, as is known in the art.

In one embodiment, display processor 112 incorporates circuitryoptimized for graphics and video processing, including, for example,video output circuitry, and constitutes a graphics processing unit(GPU). In another embodiment, display processor 112 incorporatescircuitry optimized for general purpose processing. In yet anotherembodiment, display processor 112 may be integrated with one or moreother system elements, such as the memory bridge 105, CPU 102, and I/Obridge 107 to form a system on chip (SoC). In still further embodiments,display processor 112 is omitted and software executed by CPU 102performs the functions of display processor 112.

Pixel data can be provided to display processor 112 directly from CPU102. In some embodiments of the present invention, instructions and/ordata representing a scene are provided to a render farm or a set ofserver computers, each similar to system 100, via network adapter 118 orsystem disk 114. The render farm generates one or more rendered imagesof the scene using the provided instructions and/or data. These renderedimages may be stored on computer-readable media in a digital format andoptionally returned to system 100 for display. Similarly, stereo imagepairs processed by display processor 112 may be output to other systemsfor display, stored in system disk 114, or stored on computer-readablemedia in a digital format.

Alternatively, CPU 102 provides display processor 112 with data and/orinstructions defining the desired output images, from which displayprocessor 112 generates the pixel data of one or more output images,including characterizing and/or adjusting the offset between stereoimage pairs, in the case of stereoscopic images. The data and/orinstructions defining the desired output images can be stored in systemmemory 104 or graphics memory within display processor 112. For example,CPU 102 could execute a client media player application (not shown) thatreceives a media stream from a content provider, and transmits the mediastream to the display processor 112 for viewing on the display device110. In an embodiment, display processor 112 includes 3D renderingcapabilities for generating pixel data for output images frominstructions and data defining the geometry, lighting shading,texturing, motion, and/or camera parameters for a scene. Displayprocessor 112 can further include one or more programmable executionunits capable of executing shader programs, tone mapping programs, andthe like.

CPU 102, render farm, and/or display processor 112 can employ anysurface or volume rendering technique known in the art to create one ormore rendered images from the provided data and instructions, includingrasterization, scanline rendering REYES or micropolygon rendering, raycasting, ray tracing, image-based rendering techniques, and/orcombinations of these and any other rendering or image processingtechniques known in the art.

It will be appreciated that the system shown herein is illustrative andthat variations and modifications are possible. The connection topology,including the number and arrangement of bridges, may be modified asdesired. For instance, in some embodiments, system memory 104 isconnected to CPU 102 directly rather than through a bridge, and otherdevices communicate with system memory 104 via memory bridge 105 and CPU102. In other alternative topologies display processor 112 is connectedto I/O bridge 107 or directly to CPU 102, rather than to memory bridge105. In still other embodiments, I/O bridge 107 and memory bridge 105might be integrated into a single chip. The particular components shownherein are optional; for instance, any number of add-in cards orperipheral devices might be supported. In some embodiments, switch 116is eliminated, and network adapter 118 and add-in cards 120, 121 connectdirectly to I/O bridge 107.

FIG. 2 illustrates a layout 200 of movie clips associated withtransition points, according to one embodiment of the present invention.As shown, the layout 200 includes clip layers 210 and a final program215.

The clip layers 210 illustrate individual pieces of media, such as videoor audio that may contribute to the final program 215. As shown, theclip layers 210 include layer identifiers 220 and media clips 230. Eachof the clip layers 210 may include one or more regions where an imagequality transition is preferable or not preferable.

Layer identifier 220(0) corresponds to a media clip 230(0) that includesa section where the camera performs a slow pan move, as is typicallydone to emphasize or deemphasize one or more aspects of a shot. During aslow pan, a visual artifact associated with an image quality transitionmay be relatively more noticeable. Accordingly, an image qualitytransition may not be preferred during the slow pan section of mediaclip 230(0).

Layer identifier 220(1) corresponds to a media clip 230(1) that includesa section where a product placement appears in a scene. A productplacement involves a particular product, such as a beverage, piece ofclothing, or other article, whose presence in a scene is paid for by asponsor, such as the manufacturer of the product. Because the typicalgoal of a product placement is that the placed product is viewed in afavorable light, an image quality transition may be less favored when aplaced product is visible in the scene. Accordingly, an image qualitytransition may not be preferred during the product placement section ofmedia clip 230(1).

Layer identifier 220(2) corresponds to a media clip 230(2) that includesa section where a key character appears in a scene. The director orproducer of the movie may prefer that an image quality not occur when aparticular character is featured in a scene. Accordingly, an imagequality transition may not be preferred during the key character sectionof media clip 230(2).

Layer identifier 220(3) corresponds to a media clip 230(3) that includesa section where the camera performs a whip pan move. A whip pan is atype of camera pan where the camera moves quickly in a horizontaldirection, causing heavy blurring or streaking in the scene. During awhip pan, the background scene may be relatively indistinct. As aresult, a visual artifact associated with an image quality transitionmay be relatively less noticeable during a whip pan. Accordingly, animage quality transition may be preferred during the whip pan section ofmedia clip 230(3).

Layer identifier 220(4) corresponds to a media clip 230(4) that includesa section that features a key scene in the movie. The director orproducer of the movie may prefer that an image quality not occur when aparticular scene is presented. Accordingly, an image quality transitionmay not be preferred during the key scene section of media clip 230(4).

The final program 215 illustrates the finished media that has beenedited using the clip layers 210. As shown, the final program 215includes an edit decision list (EDL) 240 and the media content 250.

The EDL 240 is used to determine which clip layer appears in the finalprogram at any given time. As such, the EDL essentially is a list ofscene changes 260, where each change includes one or more time codevalues with corresponding scene change information associated with thetime code values. For example, a scene change 260 could direct a changefrom media clip 230(0) to media clip 230(1) at a specified point intime. The EDL may include change information related to changes in anyaspect of the final program 215, including, without limitation, thevideo or film image, audio, and graphics data. In some embodiments, theEDL may be produced automatically by a media editing application. Insome embodiments, the EDL may be replaced or augmented by a timinginformation from a film log produced by a film editing application, or amanual log produced and entered by hand.

The media content 250 includes the finished media produced from the cliplayers 210 using the time code-based entries in the EDL 240. As shown,the media content 250 includes clip segments 270 that are delineated byscene changes 260. The first clip segment 270(0) of the media content250 includes a corresponding portion of media clip 230(0) associatedwith layer A. At scene change 260(0) the media content switches from thefirst clip segment 270(0) to the second clip segment 270(1) associatedwith layer B. The corresponding media clip 230(1) includes a productplacement during the entire duration of the second clip segment 270(1).

At scene change 260(1) the media content switches from the second clipsegment 270(1) to the third clip segment 270(2) associated with layer C.At scene change 260(2) the media content switches from the third clipsegment 270(2) to the fourth clip segment 270(3) associated with layerA. The corresponding media clip 230(0) includes a slow pan that beginspartially through the fourth clip segment 270(3) at time marker 280 andlasts until the end of the fourth clip segment 270(3). At scene change260(3) the media content switches from the fourth clip segment 270(3) tothe fifth clip segment 270(4) associated with layer C. The correspondingmedia clip 230(2) includes a key character during the entire duration ofthe fifth clip segment 270(4).

At scene change 260(4) the media content switches from the fifth clipsegment 270(4) to the sixth clip segment 270(5) associated with layer D.The corresponding media clip 230(3) includes a whip pan during theentire duration of the sixth clip segment 270(5). At scene change 260(5)the media content switches from the sixth clip segment 270(5) to theseventh clip segment 270(6) associated with layer E. The correspondingmedia clip 230(4) includes a key scene during the entire duration of theseventh clip segment 270(6). At scene change 260(6) the media contentswitches from the seventh clip segment 270(6) to the eighth clip segment270(7) associated with layer A.

The media content 250 may be augmented with data derived from scenechanges 260 and various characteristics of the media clips 230. Suchdata associated with the media content 250 is referred to herein asmetadata. In one embodiment, metadata corresponding to any or all scenechanges 260 may be embedded into the media content 250. Such metadatamay identify locations in the media content 250 where an image qualitytransition may be relatively less noticeable. Such locations mayidentify relatively preferred times to change image quality, if such achange is warranted. In another embodiment, metadata may be embeddedinto the media content 250, where such metadata may identify locationsin the media content 250 where an image quality transition may berelatively more noticeable, and, therefore, less preferred.

In yet another embodiment, metadata corresponding to durations of timemay be embedded into the media content 250. Such locations may identifytime spans where a change in image quality is relatively less preferredor more preferred. These durations of time may be specified via abeginning and ending time code. Alternatively these durations may bespecified via a single time code and a duration. For example, metadatacould be embedded into the media content to indicate that an imagequality transition is not preferred during clip segments 270(1), 270(4),and 270(6), corresponding to the product placement, key character, andkey scene, respectively. Metadata could also be embedded into the mediacontent to indicate that an image quality transition is not preferredduring the portion of clip segment 270(3) corresponding to the slow pan.Metadata could also be embedded into the media content to indicate thatan image quality transition is preferred during clip segment 270(5),corresponding to the whip pan.

The source for the metadata may be from any technically feasible source,including, without limitation, scene change data from an EDL, film log,or manual log; audio cues and effects; detection of particular objectsor attributes in an image sequence; and input from a director or othercontent creator.

In one embodiment, the metadata may be embedded throughout the mediacontent 250, where each metadata item is embedded at or before thecorresponding location or duration of time in the media content 250. Inanother embodiment, the metadata may be embedded in a single headersection at or near the beginning of the media content 250. In anotherembodiment, the metadata may be stored in a separate metadata file (notshown), or provided in any other technically feasible manner.

The metadata may include a reference to the relative priority orimportance of the associated location or time duration. In one example,the metadata could include a text field or an enumerated type thatindicates whether the metadata is associated with a scene change, slowpan, product placement, key character, whip pan, key scene, or someother type. In another example, the metadata could include a numericvalue that indicates the strength of preference for making, or avoiding,a change in image quality at the corresponding location or duration oftime. For example, the numeric value could vary between 0.0 and 1.0,where a higher value indicates a stronger preference for an imagequality transition versus a lower value. In such a case, a streamingmedia provider would perform an image quality transition during theduration of time associated with the higher numeric value, if feasible.

As described, the metadata may be used to determine preferred imagequality transition points, when such an image quality transition iswarranted by a change in available streaming media bandwidth or otherconditions. The metadata may be used for any other technically feasibleapplication, such as to preemptively transition to a lower image qualitylevel, resulting in a lower streaming transmission rate of the mediacontent. Such a lower transmission rate may conserve communicationchannel bandwidth, reduce cost of transmitting the streaming media, orreduce power consumption of the transmitter or receiver of the mediacontent.

FIG. 3 illustrates a transition point system 300 as may be implementedon the computing system 100 of FIG. 1, according to one embodiment ofthe invention.

As shown, the transition point system 300 includes a transition pointapplication 310, an input mechanism 320, an input data store 330, and anoutput data store 340.

The transition point application 310 is a software program includinginstructions that, when executed, performs the techniques describedabove in conjunction with FIG. 2. The transition point application 310may reside in system memory 104 where the instructions of the softwareprogram may be retrieved and executed by the CPU 102.

The input mechanism 320 receives one or more input variables, such as adescription including one or more beats, as described herein, andtransmits the one or more input variables to the transition pointapplication 310. For example, the input mechanism 320 could receive amanually entered log data, including time codes, for embedding in mediacontent. The input mechanism 320 may also be used to receive qualitativepreference data, such as whether a metadata entry is associated with ascene change, product placement, slow pan, key character, whip pan, orkey scene. The input mechanism 320 may also be used to receivequantitative preference data, such as a value ranging from 0.0 to 1.0indicating the relative preference for a transition in image qualitylevel at the time corresponding to the metadata. The input mechanism 320may provide data to the transition point application 310 via one or moreinput devices 108.

The input data store 330 provides input data to the transition pointapplication 310. As shown, the input data store 330 includes input mediacontent 332 and an edit decision list 334. The input media content 332includes an edited program such as the media content 250 of FIG. 2 priorto embedding metadata. Multiple versions of the input media content 332may be present in the input data store 330, where each versionrepresents a different encoding rate associated with a different imagequality level. In some embodiments, the multiple versions of the inputmedia content 332 may also be associated with a different sound qualitylevel of the input media content 332. The edit decision list 334includes a list of scene change data such as the EDL 240 of FIG. 2. Theinput data store 330 may also include any other input information forthe transition point application 310 including, without limitation, afilm log (not shown). The input data store 330 may be located within thesystem disk 114.

The output data store 340 receives output data from the transition pointapplication 310 for storage, streaming, or display. As shown, the outputdata store 340 includes output media content 342. The output mediacontent 342 includes the edited program from the input media content 332after the transition point application 310 has embedded metadata intothe media content. The embedded metadata may include scene changes fromthe EDL 334, a film log, or through manual input via the input mechanism320. Multiple versions of the output media content 342 at differentencoding rates may be present in the output data store 330, eachcorresponding to a different encoded rate of the input media content 332and with metadata embedded by the transition point application 310. Theoutput media content 342 may be displayed on the display device 110 forviewing. The output media content 342 may also be streamed to one ormore client media players via a communications channel.

The output data store 340 may also include any other output informationfor the transition point application 310 including, without limitation,a file (not shown) that includes metadata related to preferredtransition points, without the corresponding media content. The outputdata store 340 may be located within the system disk 114. Alternatively,the output data store 340 may be located within a memory in a formatsuitable for display on the display device 110.

FIG. 4 sets forth a flow diagram of method steps for embeddingtransition points in media content with the system of FIG. 3, accordingto one embodiment of the invention. Although the method steps aredescribed in conjunction with the systems of FIGS. 1 and 3, personsskilled in the art will understand that any system configured to performthe method steps, in any order, falls within the scope of the presentinvention.

A method 400 begins at step 402, where the transition point system 300retrieves a time marker associated with a point of interest in the mediacontent. The time marker may be associated with a specific location inthe media content or with a duration of time in the media content. Thesource of the time marker may be from any technically feasible source,including, without limitation, an edit decision list, a film log, or amanual entry. At step 404, the transition point system 300 identifies aposition in the media content corresponding to the point of interest,via any technically feasible approach. For example, the transition pointsystem 300 could identify a frame of video in the media content thatcorresponds to a time code, where the time code is equal to a time codeassociated with the time marker. A time code is a sequence of numericcodes, where each numeric code is associated with a different videoframe or film frame in the media content. In another example, thetransition point system 300 could identify a frame of video that is agiven quantity of frames from the beginning of the media content, asspecified by the time marker. At step 406, the transition point system300 embeds metadata into the media content at or before the position ofinterest. At step 408, the transition point system 300 determineswhether additional time markers remain to be embedded. If additionaltime markers remain to be embedded, then the method 400 proceeds to step402, described above. If, however, no additional time markers remain tobe embedded, then the method 400 proceeds to step 410, where thetransition point system 300 stores the media content with the embeddedtime marker data. In some embodiments, the metadata is stored in a fileseparate from the media content rather than embedded into the mediacontent. The method 400 then terminates.

FIG. 5 illustrates a content provider system 500 configured to transmita media stream as may be implemented on the computing system 100 of FIG.1, according to one embodiment of the invention. As shown, the contentprovider system 500 includes a content server 510, an input mechanism520, an input data store 530, and an output data store 540.

The content server 510 is computing device with a software programincluding instructions that, when executed, provides streaming media toone or more client media players over a communications channel. Thecontent server 510 may reside in system memory 104 where theinstructions of the software program may be retrieved and executed bythe CPU 102.

The input mechanism 520 receives one or more input variables, such asthe title of a piece of media content to transmit over a communicationschannel. The input mechanism 520 may be implemented in any technicallyfeasible manner, including, without limitation, a command lineinterface, a batch or script file, a web-based application, or agraphical user interface. The input mechanism 520 may provide data tothe content server 510 via one or more input devices 108.

The input data store 530 provides input data to the content server 510.As shown, the input data store 530 includes a high quality contentlibrary 532, a mid-level quality content library 534, and a low qualitycontent library 536. The three content libraries 532, 534, 536 includeone or more units of media content that are encoded at different datarates, or bit rates, corresponding to different image quality levels.For example, the high quality content library 532 could be encoded at2500 kilobits per second (kbps), the mid-level quality content library534 could be encoded at 300 kbps, and the low quality content library536 could be encoded at 5 kbps. In some embodiments, the multiplecontent libraries may also be associated with a different sound qualitylevel of the units of media content. Although shown with three contentlibraries at three different image quality levels, the input data store530 could include any technically feasible quantity of contentlibraries, each corresponding to a different image quality level.

The input data store 530 may also include any other input informationfor the content server 510 including, without limitation, a separatemetadata file (not shown) corresponding to media content that exists oneor more of the high quality content library 532, mid-level qualitycontent library 534, and low quality content library 536. The input datastore 530 may be located within the system disk 114.

The media stream 540 provides a mechanism to transmit media content fromthe content server 510 to one or more client media players over acommunications channel. The media streamer 540 may include a buffer (notshown) to hold media content temporarily, pending transmission to thecommunications channel.

In one example, the content server 510 could receive, via the inputmechanism 520, a command to stream specific media content. The contentserver 510 could initially select the high image quality level, andcould start reading the media content from the high quality contentlibrary 532. The content server 510 could then transmit the mediacontent via the media stream 540 to a communications channel. Thecontent server 510 could subsequently determine that the communicationchannel no longer has sufficient streaming capacity to completetransmission of the media content at the high image quality level. Thecontent server 510 could determine from metadata associated with themedia content that the communication channel has sufficient bandwidth tosustain transmission at the current image quality level until the nextpreferred transition point is reached. The content server 510 would thenwait until the time of the next preferred transition point and thenwould transition to reading media content stored on the mid-levelquality content library 534 or low-level quality content library (536).The content server 510 would begin streaming the mid-level quality mediacontent or the low-level quality media content to the media stream 540,respectively.

The content server 510 could subsequently determine that thecommunication channel no longer has sufficient streaming capacity tocomplete transmission of the media content at the mid-level imagequality level. The content server 510 could determine from metadataassociated with the media content that the communication channel doesnot have sufficient bandwidth to sustain transmission at the currentimage quality level until the next preferred transition point isreached. The content server 510 would then transition to reading mediacontent stored on the low quality content library 536 without waitinguntil the next preferred transition point. The content server 510 wouldthen being streaming the low quality media content to the media stream540.

The content server 510 could subsequently determine that thecommunication channel now has sufficient streaming capacity to completetransmission of the media content at the mid-level image quality. Thecontent server 510 would then wait until the time of the next preferredtransition point is reached and then would transition to reading mediacontent stored on the mid-level quality content library 534. The contentserver 510 would then begin streaming the mid-level quality mediacontent to the media stream 540. The process would continue untiltransmission of the selected media content completes, or thetransmission of the selected media content is terminated.

In some embodiments, deferring an image quality transition until a latertime may result in a larger image quality change at a later time. Forexample, the content provider system 400 could determine that thecommunication channel bandwidth has reduced such that the channel cansustain the current image quality level for an additional ten seconds.The content provider system 400 could immediately transition to aslightly lower image quality level to avoid an interruption in thestreaming media service. Alternatively, the content provider system 400could defer the image quality level reduction for five seconds, where apreferred transition point, such as a scene change, is identified.However, the content provider system 400 could determine that a moresignificant reduction in image quality level is now warranted in orderto avoid a service interruption. Even so, a larger reduction in imagequality level occurring at a scene change could be preferable to asmaller reduction in image quality during a critical scene.

FIGS. 6A-6B set forth a flow diagram of method steps for transmitting amedia stream that includes transition points with the system of FIG. 5,according to one embodiment of the invention. Although the method stepsare described in conjunction with the systems of FIGS. 1 and 5, personsskilled in the art will understand that any system configured to performthe method steps, in any order, falls within the scope of the presentinvention.

A method 600 begins at step 602, where the content provider system 500computes the bandwidth available to stream media content. Thecomputation may be based on a number of factors, including, withoutlimitation, dynamic characteristics of a communications channel, otherdemands on the bandwidth of the communications channel, and the bit rateof the streaming media content. At step 604, the content provider system500 selects an image quality level based on the computed bandwidth. Theimage quality level corresponds to a media content library, where themedia content stored in the media content library includes content forstreaming at a given bit rate. At step 606, the content provider system500 streams the media content at the selected image quality level.

At step 608, the content provider system 500 determines whether thebandwidth available to stream media via the communications channel haschanged. In some embodiments, the content provider system 500 may alsodetermine whether a client media player has sufficient buffer space tocontinue to stream media, based, for example, on a buffer stateassociated with the client media player. If the bandwidth has notchanged, and, in some embodiments, the client media player hassufficient buffer space, then the method 600 proceeds to step 606,described above. If, however, the bandwidth has changed, or, in someembodiments, the client media player does not have sufficient bufferspace, then the method 600 proceeds to step 610, where the contentprovider system 500 determines whether media content is available at anappropriate quality level. For example, if the bandwidth available tostream media content has decreased, then the content provider system 500would determine whether media content is available at a lower qualitylevel, corresponding to a lower bit rate. If the bandwidth available tostream media content has increased, then the content provider system 500would determine whether media content is available at a higher qualitylevel, corresponding to a higher bit rate. If media content is notavailable at an appropriate quality level, then the method 600 proceedsto step 606, described above.

If, however, media content is available at an appropriate quality level,then the method 600 proceeds to step 612, where the content providersystem 500 computes the time until the next preferred transition point,as determined from metadata embedded in the media content or storedseparately from the media content. At step 614, the content providersystem 500 determines whether the bandwidth available to stream mediacontent is sufficient that the content provider system 500 may waituntil the preferred transition point without incurring a serviceinterruption. If the bandwidth available to stream media content issufficient, then the method 600 proceeds to step 616, where the contentprovider system 500 waits until the next preferred transition point inthe media content being streamed.

At step 618, the content provider system 500 selects a new image qualitylevel based on the current bandwidth available for streaming mediacontent. At step 620, the content provider system 500 streams the mediacontent at the newly selected image quality level. The method 600 thenproceeds to step 608, described above. The content provider system 500continues to stream the media content via the communications channel,adaptively transition quality level as appropriate, until all of themedia content is streamed, or streaming is interrupted for a particularreason, such as a command to terminate streaming received from a user.

Returning to step 614, if the bandwidth available to stream mediacontent is not sufficient, then the method 600 proceeds to step 618,described above.

It will be appreciated that the system shown herein is illustrative andthat variations and modifications are possible. For example, theapproaches disclosed herein are described in the context of streamingmedia. However, these approaches could be implemented in any environmentwhere the bandwidth available for media content is adaptive to dynamicchanges in a communications channel or where limited resources areavailable for decoding or rendering a stream of data. In anotherexample, the content provider could look forward into the media contentin order to predict the next two, three, or more quality transitionpoints. The content provider would not be restricted to select only thenext quality transition point.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder or out of order, depending upon the functionality involved. Itwill also be noted that each block of the block diagrams and/orflowchart illustration, and combinations of blocks in the block diagramsand/or flowchart illustration, can be implemented by special purposehardware-based systems that perform the specified functions or acts, orcombinations of special purpose hardware and computer instructions.

In sum, transition points are embedded in streaming media, where thetransition points are accessible to a content provider. The transitionpoints may be manually generated or automatically generated.Alternatively, the transition points may be any combination of manuallygenerated and automatically generated transition points. The transitionpoints provide guidance to a content provider system as to when totransition from one quality level to another quality level. Suchtransition points may identify either a point in time or a time range.Transition points may identify point or range in time where a transitionin image or sound quality level is preferred. Alternatively, transitionpoints may identify point or range in time where a transition in imageor sound quality level is not preferred. If a content provider systemdetects an impending change in available communications channelbandwidth, then the content provider system analyzes transition pointsnear the time of the impending bandwidth change. If feasible, thecontent provider system changes the quality level of the streaming mediaat an appropriate transition point. As a result, quality level changesoccur at less noticeable points in the streaming media, rather than atmore critical points, thus improving the viewing experience of theend-user.

While the foregoing is directed to embodiments of the present invention,other and further embodiments of the invention may be devised withoutdeparting from the basic scope thereof, and the scope thereof isdetermined by the claims that follow.

What is claimed is:
 1. A computer-implemented method to reducenoticeability of quality level changes during streaming of mediacontent, based on transition points, the computer-implemented methodcomprising: providing a predefined set of values ranging from explicitlycharacterizing weakest candidacy as a transition point to explicitlycharacterizing strongest candidacy as a transition point; retrieving atime marker associated with a point of interest in the media content;identifying a first position within the media content corresponding tothe point of interest; generating, by operation of one or more computerprocessors, a transition marker specifying a value that explicitlycharacterizes a candidacy of the point of interest as a transitionpoint, wherein the specified value is selected from the predefined setof values, wherein the transition marker is embedded into the mediacontent at a second position that is no later in time than the firstposition; and during streaming of the media content, causing a clientmedia player to transition from an initial image quality level to atarget image quality level at a point in time determined based on thetransition marker, in order to reduce noticeability of the transition,wherein the target image quality level is determined based on a measureof available bandwidth.
 2. The computer-implemented method of claim 1,wherein the first position is equal to the second position.
 3. Thecomputer-implemented method of claim 1, wherein the transition markerspecifies a location within the media content where a transition ispreferred.
 4. The computer-implemented method of claim 1, wherein thetransition marker specifies a duration of time within the media contentwhere a transition is not preferred.
 5. The computer-implemented methodof claim 1, wherein retrieving the time marker comprises receiving thetime marker from a user.
 6. The computer-implemented method of claim 1,wherein the initial image quality level comprises a first image qualitylevel, wherein the target image quality level comprises a second imagequality level, wherein the transition marker is generated by a contentserver operatively connected to the client media player via acommunications channel for streaming the media content, wherein thetransition marker further specifies an enumerated type explicitlycharacterizing whether the point of interest is a scene change, slowpan, product placement, key character, whip pan, or key scene; whereineach enumerated type, selected from a scene change, slow pan, productplacement, key character, whip pan, and key scene, is specifiable in thetransition marker; wherein the second image quality level is differentfrom the first image quality level, wherein the second image qualitylevel is one of: (i) higher than the first image quality level and (ii)lower than the first image quality level; wherein the predefined set ofvalues comprises a predefined set of numerical values ranging fromexplicitly characterizing weakest candidacy as a transition point toexplicitly characterizing strongest candidacy as a transition point,wherein the specified value comprises one of the predefined set ofnumerical values.
 7. The computer-implemented method of claim 6, whereinthe computer-implemented method further comprises: determining a measureof available bandwidth, based on at least two of: (i) a maximum capacityof the communications channel; (ii) a measure of current utilization ofbandwidth of the communications channel; (iii) and a bit rate of themedia content; preemptively transitioning to a first lower image qualitylevel based on the transition marker and despite sufficient bandwidthand processing power being available to avoid transitioning to the firstlower image quality level, in order to reduce power consumption andtransmission cost associated with streaming the media content; whereinthe client media player is caused to: in transitioning between imagequality levels during streaming of the media content, prioritize alarger reduction in image quality level occurring at higher-candidacytransition point over a smaller reduction in image quality leveloccurring at a lower-candidacy transition point.
 8. Thecomputer-implemented method of claim 7, wherein the first position isidentified based on a time code corresponding to the time marker;wherein each content type, selected from: (i) a cut point within themedia content; (ii) an object being visible in a scene of the mediacontent; and (iii) a type of camera move; is identifiable by thetransition marker; wherein the computer-implemented method furthercomprises: storing the media content on a computer-readable medium,wherein the media content comprises an output of the retrieving,identifying, and generating steps; wherein the client media player isoperable to: determine a measure of available buffer space on the clientmedia player; and upon determining that the measure of available bufferspace falls below a predefined threshold, transition to a second lowerimage quality level despite sufficient bandwidth and processing powerbeing available to avoid transitioning to the second lower image qualitylevel.
 9. The computer-implemented method of claim 8, wherein eachtarget type selected from: (i) a location within the media content wherea transition is preferred; and (ii) a duration of time within the mediacontent where a transition is not preferred; is specifiable in thetransition marker; wherein the time marker is generable based on eachmarker element selected from: (i) an edit decision list; (ii) theenumerated type; (iii) audio cues from the media content; and (iv)detection of specified objects in the media content.
 10. Thecomputer-implemented method of claim 9, wherein the predefined set ofvalues includes at least three distinct values, wherein the firstposition is one of: (ii) equal to the second position and (ii) later intime than the second position; wherein the time marker is generatedbased on: (i) the edit decision list; (ii) the enumerated type; (iii)the audio cues from the media content; and (iv) the detection of thespecified objects in the media content.
 11. A non-transitorycomputer-readable medium including instructions executable to perform anoperation to reduce noticeability of quality level changes duringstreaming of media content, based on transition points, the operationcomprising: providing a predefined set of values ranging from explicitlycharacterizing weakest candidacy as a transition point to explicitlycharacterizing strongest candidacy as a transition point; retrieving atime marker associated with a point of interest in the media content;identifying a first position within the media content corresponding tothe point of interest; generating, by operation of one or more computerprocessors when executing the instructions, a transition markerspecifying a value that explicitly characterizes a candidacy of thepoint of interest as a transition point, wherein the specified value isselected from the predefined set of values, wherein the transitionmarker is embedded into the media content at a second position that isno later in time than the first position; and during streaming of themedia content, causing a client media player to transition from aninitial image quality level to a target image quality level at a pointin time determined based on the transition marker, in order to reducenoticeability of the transition, wherein the target image quality levelis determined based on a measure of available bandwidth.
 12. Thenon-transitory computer-readable medium of claim 11, wherein the firstposition is equal to the second position.
 13. The non-transitorycomputer-readable medium of claim 11, wherein the transition markerspecifies a location within the media content where a transition ispreferred.
 14. The non-transitory computer-readable medium of claim 11,wherein the transition marker specifies a duration of time within themedia content where a transition is not preferred.
 15. Thenon-transitory computer-readable medium of claim 11, wherein retrievingthe time marker comprises receiving the time marker from a user.
 16. Acomputing system to reduce noticeability of quality level changes duringstreaming of media content, based on transition points, the computingsystem comprising: a memory that is configured to store instructions fora program; and a processor that is configured to execute theinstructions for the program to perform an operation comprising:providing a predefined set of values ranging from explicitlycharacterizing weakest candidacy as a transition point to explicitlycharacterizing strongest candidacy as a transition point; retrieving atime marker associated with a point of interest in the media content;identifying a first position within the media content corresponding tothe point of interest; generating a transition marker specifying a valuethat explicitly characterizes a candidacy of the point of interest as atransition point, wherein the specified value is selected from thepredefined set of values, wherein the transition marker is embedded intothe media content at a second position that is no later in time than thefirst position; and during streaming of the media content, causing aclient media player to transition from an initial image quality level toa target image quality level at a point in time determined based on thetransition marker, in order to reduce noticeability of the transition,wherein the target image quality level is determined based on a measureof available bandwidth.
 17. The computing system of claim 16, whereinthe first position is equal to the second position.
 18. The computingsystem of claim 16, wherein the transition marker specifies a locationwithin the media content where a transition is preferred.
 19. Thecomputing system of claim 16, wherein the transition marker specifies aduration of time within the media content where a transition is notpreferred.
 20. The computing system of claim 16, wherein retrieving thetime marker comprises receiving the time marker from a user.